feat(local-model): list every model that fits, and add Muse Glimmer 30B and Ornith 1.0 9B - #803

What this changes

"Run a local model" (the window that installs a coding model on the Mac the Codeman server runs on) picks from a compiled catalogue and showed its pick plus at most four other models. This PR gives people more to choose from:

1. The chooser lists every catalogue model that fits the Mac, not five. MAX_ALTERNATIVES goes from 4 to 11, above what the catalogue holds, and a new test fails if the catalogue ever outgrows it, so no model can drop off the list unseen. The ranking rules are unchanged.
2. Two new models in three rows, none "Recommended" (that word stays reserved for a pair we ran on a real Mac). The first two rows are estimated; the third was added after a run on a real Mac and carries measured figures (see "Run on a real Mac" below):

3. The "set aside on purpose" list was read again at its sources on 2026-11-10 and now says why each model is still out (for example ripple-preview:65b, published 2026-11-09, needs runtime 0.40.16, one above the pin).

What a Mac is offered now

Only two memory sizes get a different pick. Every other pick is unchanged; each Mac just sees a longer list under it, and from 48 GB up that list has Muse Glimmer 30B as its MLX build too (never the pick while the disk has room for Qwen3.6 35B-A3B. Where it has not, on a Mac of 48 GB or more with between 29.8 and 33.4 GB free, the MLX row is now the pick where North Mini Code was: its speed grades ok and its rank is higher. One existing test expectation changed for that.)

The Ornith row is its own commit (d25cc086), so it can be dropped without touching the rest if a 16 GB answer is not wanted yet.

How the new rows were derived

• Sizes, the runner and the requires version are what the registry serves a Mac today. npm run check:local-model-catalog confirms all eleven rows byte for byte.
• Memory figures of the two GGUF rows are computed from each model's config.json by the llama.cpp rule in the catalogue header. The MLX row's are measured (below). test/local-model-catalog.test.ts holds each architecture and recomputes every figure, and refuses a row it does not know.
• Whether a build has a multi-token-prediction layer was read from its GGUF header (a ranged read of the first megabytes, never the weights).

Run on a real Mac (2026-11-12)

A MacBook Pro (M5 Pro, 48 GB, macOS 27.2) with the pinned runtime, Ollama 0.40.15. Its Codeman ran the tree of this stack (#803 to #808) merged onto 1.50.1. The window was driven by real clicks, from a Chromium on another machine through an SSH forward to the Mac's own server, so every state on screen was the Mac's.

• The list: the chooser offered nine models on that Mac (the pick and eight more), both new rows among them.

• Ornith 1.0 9B, installed through the window ("Add a model"):

  • 5,629,110,568 bytes pulled, exactly the catalogue's figure, in 88 seconds;
  • a GGUF at Q4_K_M; loaded at 131,072 tokens it took 7.5 GB, all of it on the GPU;
  • the tool call check passed, and an OpenCode session on it answered.

• Muse Glimmer 30B, installed with the row's own total:

  • 18,157,010,252 bytes pulled, exactly the catalogue's figure, in 266 seconds;
  • a GGUF at Q4_K_M (27.9B parameters); loaded at 131,072 tokens it took 16.0 GB, all of it on the GPU;
  • the tool call check passed.

• Muse Glimmer's speed, measured on the M5 Pro through the runtime's own API: 15.2 and 14.5 tokens per second in two runs (the catalogue's estimate for that chip is 9.1), and a prompt of 8,465 tokens read in 25 seconds.

• Muse Glimmer in agent use is much slower than those figures suggest. Through OpenCode a one-line task (sent with the skill's sendmark) did not finish within 7 minutes, in two separate runs. In the second one the pane was read once a minute: nothing for four minutes, then "Thinking" until the limit. No error showed, the model stayed loaded, and the same task took 18 seconds on Ornith 1.0 9B. The likely cause is the model's own reasoning at that speed, with OpenCode's title request served first on the one slot; this was not taken further. On that Mac the row is tagged "Slow" and is not the pick. It IS the pick on a 32 GB Mac with a Max chip (the table below: computed at 11.8 tokens per second there, graded ok), and no such Mac was available to run it on. Whether the GGUF build should stay that Mac's pick after this result is a decision for the maintainer.

• The MLX build of the same model (muse-glimmer:30b-mlx, the third row of this PR, commits 5396e33b and 44239499), on a second instance of the pinned runtime on the same Mac:

  • 19,053,621,992 bytes pulled (the catalogue's figure), 19.1 GB loaded at 131,072 tokens;
  • 30.2 tokens per second on the same 300-token request the GGUF build wrote 15.2 on, and 32.6 for a 1,500-token answer. The tag carries a draft model: the runtime logged speculative decoding, with 0.34 to 0.62 of the drafts accepted;
  • a prompt of 8,465 tokens read in 17 seconds, and a correct tool call in under 3 seconds;
  • an OpenCode turn on a one-line prompt finished in 45 seconds. The GGUF build had shown no answer after 7 minutes on a one-line task, which is not the same prompt: that one went through the skill's sendmark, which adds its answer-file sentence.

• Its memory, measured from the runner's own log line after every request (memory peak=... held=...) and the process footprint at the end:

  • what stayed held grew by 37,900 to 45,100 bytes per token between prompts of 30,000, 60,000 and 94,600 tokens (the GGUF build computes to 7,072). That is close to every layer's cache at full length, so the MLX runner evidently does not trim the sliding-window layers; this is an inference from the figure, its source was not read;
  • twelve separate prompts of 15,000 tokens added 0.53 GiB of prefix snapshots each until the runner dropped the oldest, so the row counts the whole 8 GiB;
  • the process stood 0.2 GB above the runner's own count at the end. Its lifetime peak stood 3.0 to 3.3 GB above that, which is only a floor for the peak of a long prompt: a larger transient during the earlier, longer prompts would not show in a maximum read at the end.

  The row takes 45,000 bytes per token, 8 GiB + 0.5 GB, and for the peak the first measured row's 5 GB. That is 36.1 GB at 65,536 tokens (0.960 of a 48 GB Mac's estimated budget) and 39.0 GB at 131,072 (0.972 of the 40.2 GB that Mac's runtime measures).

• Speed in the catalogue: 5.4 tokens per second per 100 GB/s, which is 0.32 of the Qwen3.6 35B-A3B row's 16.9, because the two were measured side by side with one request (32.6 against 101.9). Like that row's figure it is on the low side: it computes to 16.6 on the Mac that wrote 32.6.

• The two open runtime issues on this tag, read again: 17656 says its nvfp4 weights cannot be accelerated by the MLX engine (on this Mac the runtime logged the MLX engine on the GPU, and the build was twice as fast as the GGUF one); 18269 reports it stuck, and a restart, on a 32 GB Mac (the measured figures keep the row off every Mac below 48 GB, and a test pins that; whether the memory fix its maintainer promised predates the pinned runtime is not known).

• The same session on the installed Qwen3.6 35B-A3B (the measurement its row owed): 39,600 bytes held per token, and twelve separate prompts added 0.5 GiB of prefix snapshots each, 5.6 GiB in all and still growing. The run ended at 33.4 GB of the 40.2 GB that Mac's runtime measures. Its row's figures are not changed here, and they do not count those snapshots: whether they should is a decision for the maintainer.

• An existing row, Qwen3.5 9B, was installed by the 1.50.1 release on the same Mac first: 6,550,825,550 bytes, the catalogue's figure.

Not run: every other row, and any Mac but this one. The recommendation a 16 GB Mac gets (Ornith 1.0 9B) was not seen on a 16 GB Mac. The MLX row was not installed through the window (that would have been a fourth download of 19 GB): its pull, load, tool call and memory were measured on a second instance of the same runtime binary.

Known limits, stated plainly

• Measured on one Mac only (above). What every OTHER Mac gets from the two rows is still a calculation, and so is each row's peak memory with the whole context in use. The install's own check after the download (the whole model on the GPU at the chosen context, one tool call) is what catches a load judged too kindly.
• Benchmarks are the vendors' own cards.
• Runtime issue 16992 (open) reports tool calls arriving as text in long agent chains, filed on Ornith 1.0 35B. Its one diagnosed cause sits in the tool parser the pinned runtime uses for every Qwen-family model, so it is not new with this PR: the rows already shipped share it. It is now documented in the catalogue header and in docs/local-models.md. Ornith 1.0 9B is ranked under MiMo for that reason, so it is the pick only where nothing else fits.
• Ollama 0.40.16 is out; the runtime pin stays at 0.40.15 here. Moving it needs an install on a Mac first.

Testing

• CI on this PR is the full gate.
• 23 targeted test files, 1,327 tests on the final tree: every test/local-model-*.test.ts, both local model route tests and the docs tests. The combined tree with feat(local-model): a library of models on this Mac, with sizes, delete, add and switch #804 to docs(wiki): Local Models, the walkthrough from install to cloud plus local #808 passes its own set too (30 files, 1,713 tests).
• New for the MLX row: its figures are pinned as the second measured row, a test asserts that it counts the whole prefix budget and that its per-token figure lies between the full-attention layers alone and all layers at full length, and a sweep over every Mac size asserts that it is offered from 48 GB only and is never the pick of a Mac whose disk has room.
• npm run typecheck, npm run lint, npm run format:check: clean.
• npm run check:local-model-catalog: all eleven rows match the registry; the only drift it reports is the newer runtime release.
• Not run: test/local-model-ui.browser.test.ts (the window's code is unchanged in this PR).

Files

src/config/local-model-catalog.ts, src/local-model/recommend.ts, a comment in src/types/local-model.ts, the tests test/local-model-catalog.test.ts, test/local-model-recommend.test.ts, test/local-model-fits-budget.test.ts, and the docs docs/local-models.md, docs/api-reference.md, CLAUDE.md.